Workshop Session 1

class: nocopy
layout: true

.remark-slide-content {
  padding-top: 15px;
  padding-left: 50px;
  padding-right: 50px;
  padding-bottom: 50px;
}

.remark-slide-content > h1:first-of-type {
  margin-top: 10px;
  margin-bottom: 25px;
}

.remark-slide-content > h2:first-of-type {
  margin-top: 10px;
  margin-bottom: 25px;
}

.remark-slide-content > h3:first-of-type {
  margin-top: 10px;
  margin-bottom: 10px;
}

.reduced_opacity {
  opacity: 0.1;
}

.pull-left-45{
  float: left;
  width: 42%
}

.pull-right-55{
  float: right;
  width: 52%
}

.bg_aiu {
  position: relative;
  z-index: 1;
}

.bg_aiu::before {    
      content: "";
      background-image: url("assets/Snowy_aiu.jpg");
      background-size: cover;
      position: absolute;
      top: 0px;
      right: 0px;
      bottom: 0px;
      left: 0px;
      opacity: 0.4;
      z-index: -1;
}

.bg_a1 {
  position: relative;
  z-index: 1;
}

.bg_a1::before {    
      content: "";
      background-image: url(https://source.unsplash.com/lEWEQyEN8mU/1600x900);
      background-size: cover;
      position: absolute;
      top: 0px;
      right: 0px;
      bottom: 0px;
      left: 0px;
      opacity: 0.4;
      z-index: -1;
}

.remark-code, .remark-inline-code {font-size: 60%;
}

.big {
  font-size: 1.5em;
  color: CornFlowerBlue;
}

.fade1 {
  opacity: 0.66;
}

.fade2 {
  opacity: 0.7;
}

blockquote {
  border-left: solid 5px #b31b1b;
  padding-left: 1em;
}

.footnote {
  position: absolute;
  bottom: 60px;
  padding-right: 4em;
  font-size: 0.5em;
}
</style>

---
class: left, bg_aiu

<BR/><BR/><BR/><BR/><BR/><BR/>
## ECN339 Finance Data Workshop

<BR/>

### Introduction +<br/>R, Linear Algebra, Finance Terms, Nikkei Needs
#### [Clinton Watkins](https://www.clintonwatkins.com)<BR/>Global Business Program, Akita International University
#### Winter 2022

---

## Intro to the course

### Objectives

1.	Develop the ability to conduct empirical analyses and interpret results.

2.	Gain an understanding of stock market data and selected financial models.

3.	Gain familiarity with the methodology of empirical research and coding in R.<p/>

### Useful links

- [ECN339 AIMS Page](https://aims.aiu.ac.jp/course/view.php?id=4974)

- [ECN339 Slack Workspace invite](https://join.slack.com/t/ecn339w2022/shared_invite/zt-100i0j7fr-SZ6I9NWl2JToVelE_IRPOg)

- [Setting up R and RStudio](https://www.clintonwatkins.com/courses/fd/fdw_r_intro_nb_20211218.nb.html). Hopefully you have already done this.<p/>

- These Workshop slides:
    - [https://www.clintonwatkins.com/courses/fd/1/Workshop-1.html](https://www.clintonwatkins.com/courses/fd/1/Workshop-1.html).

---

## Syllabus

- [Online syllabus](https://www.clintonwatkins.com/courses/fd/syllabus_websitev_20211119.html) on my website and on AIMS.<p/>

---

## Topics

> We will investigate and model the cross-section of Japanese stock returns with R and RStudio using Nikkei Needs financial data on Tokyo Stock Exchange listed firms.<sup>1<sup/>

1. Introduction
    - R and RStudio, basics of linear algebra, OLS estimator.
    - Finance terms, Nikkei Needs Japanese stocks database.

2. Topic 1: Factor Models
    - Market model, Fama-French models, principal components.
    
3. Topic 2: Fama Macbeth model for panel data
    - Estimating risk premia.
    
4. Topic 3: Bankruptcy Risk and quantile portfolios
    - Z-score, O-score

5. Wrap-up and Quiz.

.footnote[[1] We will approach the course topics flexibly and they may be adjusted depending on our pace through the course.]
---

## R and RStudio

- R is a powerful language suited to statistical analysis, graphics and many other applications.  
- RStudio provides a convenient interface for using R.

- Both are free. We will use R and RStudio for this course.

- Hopefully you have already downloaded and set up both using the [guide on my website](https://www.clintonwatkins.com/courses/fd/fdw_r_intro_nb_20211218.nb.html).

- [R statistical programming language <img src="data:image/png;base64,#assets/Rlogo.png" height="20px" />](https://www.r-project.org/).
    
    - [RStudio <img src="data:image/png;base64,#assets/rstudio-icon.png" height="20px"/>](https://www.rstudio.com/) and [RStudio Cloud](https://rstudio.cloud/).
    
<center><img src="data:image/png;base64,#assets/rstudio1.jpg" height="250px"/></center>

???

Please download and install this software for the course.

---
class: center middle

.bg-washed-green.b--dark-green.ba.bw2.br3.shadow-5.ph4.mt5[
Important in this workshop course:

Enter and execute the code for the analyses in RStudio as we go through them.

If you encounter an error, do not hesitate to ask me for help.
]

---

## If you haven't set up RStudio yet...

- Use [RStudio Cloud](https://rstudio.cloud/) for this session.

- Make a free [RStudio Cloud](https://rstudio.cloud/) account now (provides limited free monthly computing hours, likely not enough for this course).
    
    - Create a new project for your work in this course.
    
    - You will probably need to load some packages.
    
    - Do not forget to download any files you create for use later on your own computer.
    
- Set up R and RStudio on your computer by the next class and do the pre-assignment.

---

## R and RStudio [set-up guide](https://www.clintonwatkins.com/courses/fd/fdw_r_intro_nb_20211218.nb.html)

---

## Resources for R

.pull-left[
### Online guides

- A comprehensive [introduction to R](https://cran.r-project.org/doc/manuals/r-release/R-intro.pdf).

- A textbook [Introduction to Econometrics with R](https://www.econometrics-with-r.org/).

- [Documentation and guides for various applications contributed by R users](https://cran.r-project.org/other-docs.html#english).

- [Massimiliano's R Page on GitHub](https://github.com/massimilianoj/R_beginners) contains [R for Beginners](https://github.com/massimilianoj/R_beginners/blob/master/R_beginners.pdf), used in my econometrics, finance and macroeconomics classes in Kobe-U.

- [Massimiliano](https://link.springer.com/book/10.1007/978-3-030-34529-7) worked for me as a TA and RA in Kobe University. We are working on a book about analysing foreign exchange markets using R.

]

.pull-right[

### CRAN Task Views

- The [Comprehensive R Archive Network](https://cran.r-project.org/) (CRAN) Task Views provide information about packages that can be used with R.

- Packages extend the functionality of R. They provide routines for various types of data manipulation, analysis, graphics and visualisation, econometric models, and many more purposes.

- Packages are constantly being developed and updated by R users.

- The most relevant Task View for this course is [Empirical Finance](https://CRAN.R-project.org/view=Finance)

]

---

## Resources for R

.pull-left[

### User community
- R is an open source language developed by users and has a huge online community.

- A good place to look for Q&A and help is [stackoverflow](https://stackoverflow.com/questions/tagged/r).

### Google is your friend

- If you are stuck, [Google search](https://www.google.com/) for information that will help you answer your question.

- [R Seek <img src="data:image/png;base64,#assets/Rlogoseek.jpg" height="20px"/>](https://rseek.org/) is a search engine for R.

]

.pull-right[
### R-Bloggers

- [<img src="data:image/png;base64,#assets/R_02.jpg" height="20px"/> R-Bloggers](https://www.r-bloggers.com/) contains many useful posts and [Tutorials](https://www.r-bloggers.com/how-to-learn-r-2/) on how to use R for a wide variety of applications (not just finance).

### Books

- There are many good books on using R.

- Tsay (2014), Ang (2021) and Bennett and Hugen (2016) for finance.
    - Kleiber and Zeileis (2008) and Pfaff (2008) for econometrics.
    - Matloff (2011) is good for general programming in R.

]

---

```r
library(rfishdraw)
file <- system.file("extdata",package = "rfishdraw")
dat <- readRDS(paste0(file,"/","output_json.rds"))
for (i in seq(length(dat))) {dat[[i]] <- cbind(dat[[i]],i)}
plot(NA, xlim = c(0,520),ylim = c(-320,0),axes = F,xlab = " ",ylab = " ")
for(i in seq(length(dat))) {lines(x = dat[[i]][,1],
                                  y = -dat[[i]][,2], lwd=2, col = "blue")}
```

---

## A map of ...

```r
library(leaflet)
leaflet() %>% addTiles() %>% 
  setView(140.19838192985657, 39.62662109562309, zoom = 17)
```

<div id="htmlwidget-4737416437283d23d944" style="width:100%;height:424.8px;" class="leaflet html-widget"></div>
<script type="application/json" data-for="htmlwidget-4737416437283d23d944">{"x":{"options":{"crs":{"crsClass":"L.CRS.EPSG3857","code":null,"proj4def":null,"projectedBounds":null,"options":{}}},"calls":[{"method":"addTiles","args":["//{s}.tile.openstreetmap.org/{z}/{x}/{y}.png",null,null,{"minZoom":0,"maxZoom":18,"tileSize":256,"subdomains":"abc","errorTileUrl":"","tms":false,"noWrap":false,"zoomOffset":0,"zoomReverse":false,"opacity":1,"zIndex":1,"detectRetina":false,"attribution":"© <a href=\"http://openstreetmap.org\">OpenStreetMap<\/a> contributors, <a href=\"http://creativecommons.org/licenses/by-sa/2.0/\">CC-BY-SA<\/a>"}]}],"setView":[[39.6266210956231,140.198381929857],17,[]]},"evals":[],"jsHooks":[]}</script>

---
class: middle

### [Conway's Game of Life](https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life) [(R-Bloggers)](https://www.r-bloggers.com/2012/11/fast-conways-game-of-life-in-r/)

.pull-left-45[

<BR/>

]

.pull-right-55[

```r
library(caTools)
side=500
steps=300
X <- matrix(nrow=side, ncol=side)
X[] <- rbinom(side^2,1,0.4)
storage <- array(0, c(side, side, steps))
# the simulation                                             
for (i in 1:steps)
{
  allW = cbind( rep(0,side), X[,-side])
  allNW = rbind(rep(0,side),
          cbind(rep(0,side-1),X[-side,-side]))
  allN = rbind(rep(0,side),X[-side,])
  allNE = rbind(rep(0,side),
          cbind(X[-side,-1],rep(0,side-1)))
  allE =  cbind(X[,-1],rep(0,side))
  allSE = rbind(cbind(X[-1,-1],
          rep(0,side-1)),rep(0,side))
  allS = rbind(X[-1,],rep(0,side))
  allSW = rbind(cbind(rep(0,side-1),
          X[-1,-side]),rep(0,side))
  X2 <- allW + allNW + allN + allNE +
    allE + allSE + allS + allSW
  X3 <- X
  X3[X==0 & X2==3] <- 1
  X3[X==1 & X2<2] <- 0
  X3[X==1 & X2>3] <- 0
  X <- X3
  storage[,,i] <- X2
}
storage <- storage/max(storage)
write.gif(storage, "conway.gif", 
          col="jet", delay=5)
```

]

---

### Build websites using R blogdown

---

### Shiny apps built using R

---

### Shiny app examples...

---

### Top 8 things to know about R

1. R is an *object oriented language*. Everything in R is an object with particular properties that can interact with other objects in certain ways. Understanding the different types of objects is key to using them. E.g. vector, matrix, data frame, list, function.

2. The *assignment operator* is used to assign values to an object.
    
    ```r
    a <- 2
    b <- 3
    a+b
    ```
    
    ```
    ## [1] 5
    ```

3. *Packages* extend the functionality of base R. Packages contain code (functions), data and documents. To use a package it must be installed (once) and then loaded each session when you want to use it. Find information about packages from Task Views or internet search.
    
    ```r
    install.packages("leaflet")
    library(leaflet)
    ```

4. When you have an *error* executing your code, 99% of the time it will be because (i) you mistyped the code or (ii) you have not loaded a package required for a function to work.

---

### Top 8 things to know about R

<ol start=5>
  <li> The *working directory* is where R will look for and save files.

```r
getwd()
```

```
## [1] "/Users/clinton/Dropbox/syncd_r_data/FDW/lectures/1/lecture"
```

```r
setwd("/Users/clinton/Dropbox/syncd_r_data/FDW/preparation")
```
  </li>
  
  <li> Lots of documentation and help is available within R. Try:

```r
?seq
help(seq)
example(seq)
help(package - AER)
help (topic = "Arithmetic")
help.search("regression")
??regression
```
  </li><p/>
  
  <li> *Reproducible research:* do your work in an R notebook, including explanation, code and output. This way your work can be explained to tohers and will be reproducible later, even when you may have forgotten exactly what you did. 
  </li><BR/>
  
  <li> Practice *writing code* as much as possible. It will be frustrating at first, but your skills will improve rapidly if you keep at it.
  </li>
<ol>

---
class: center middle

.bg-washed-green.b--dark-green.ba.bw2.br3.shadow-5.ph4.mt5[
The only way to write good code is to write tons of bad code first. 
Feeling shame about bad code stops you from getting to good code.

.tr[
— Hadley Wickham
]]

---

### R Notebooks

.pull-left[

- I've uploaded an [R Notebook template](https://www.clintonwatkins.com/courses/fd/1/Rmd-template.zip) to my website and AIMS showing many of the useful features.

- Download the template and use during the class and for your homework.

- Open a new Notebook from the menu at the top-right of the RStudio toolbar or write in the template I provided.

- A Notebook consists of the YAML header, text, code chunks and output.

- Cut, paste and modify examples from the template.

- Saving the .Rmd file produces or updates the .html file Notebook.

- Remember to run all code (menu: Run All) before saving your Notebook.

]

.pull-right[

***

]

---

### Basics of linear algebra in R

- Knowing some basics of linear algebra will help you understand both how your R code works and the theory behind the financial econometric models that we will cover.

- A scalar object that represents a single value (numeric, character, logical).

- Assigning a random scalar:
    
    ```r
    a <- sample(1:10, 1)
    a
    ```
    
    ```
    ## [1] 4
    ```

- A vector is a sequence of elements of the same data type (numeric, character, logical).

`$$x = \begin{bmatrix}
x_1\\
x_2\\
\vdots\\
x_n
\end{bmatrix}$$`

- A vector may be a row vector or column vector (like `$x$` above).

- Vector arithmetic is done element by element, or member-wise.

---

### Vector arithmetic

.pull-left[

- Create two vectors by concatenation:
    
    ```r
    a <- c(1, 3, 5, 7)
    b <- c(1, 2, 4, 8)
    ```

- Arithmetic with scalars:
    
    ```r
    c <- 2 * a + 3
    c
    ```
    
    ```
    ## [1]  5  9 13 17
    ```

- Addition:
    
    ```r
    a + b
    ```
    
    ```
    ## [1]  2  5  9 15
    ```

- Subtraction:
    
    ```r
    a - b
    ```
    
    ```
    ## [1]  0  1  1 -1
    ```
]

.pull-right[
- Multiplication:
    
    ```r
    a * b
    ```
    
    ```
    ## [1]  1  6 20 56
    ```

- Division:
    
    ```r
    a / b
    ```
    
    ```
    ## [1] 1.000 1.500 1.250 0.875
    ```

- What is the length of the vector?
    
    ```r
    length(a)
    ```
    
    ```
    ## [1] 4
    ```

- Recycling rule: where vectors are of unequal length, the shorter is recycled to match the length of the longer.
    
    ```r
    d <- c(1, 3, 5, 7, 9, 11, 13, 15)
    a + d
    ```
    
    ```
    ## [1]  2  6 10 14 10 14 18 22
    ```

]

---

### Vectors

.pull-left[

- Create another vector:
    
    ```r
    x <- c(1.8, 3.14, 4, 88.169, 13)
    x
    ```
    
    ```
    ## [1]  1.800  3.140  4.000 88.169 13.000
    ```

- Subsetting the first and fourth elements using square brackets:
    
    ```r
    x[c(1,4)]
    ```
    
    ```
    ## [1]  1.800 88.169
    ```

- Or subset by excluding elements:
    
    ```r
    x[-c(2,3,5)]
    ```
    
    ```
    ## [1]  1.800 88.169
    ```

- We can use various operators on vectors, e.g.:
    
    ```r
    round(log(x),3)
    ```
    
    ```
    ## [1] 0.588 1.144 1.386 4.479 2.565
    ```
    
]

.pull-right[ 
- Joining vectors:
    
    ```r
    e <- c(a,b)
    e
    ```
    
    ```
    ## [1] 1 3 5 7 1 2 4 8
    ```
  
- A vector or matrix can be numeric, logical, character. All elements must be the same type.
    
    ```r
    mode(x)
    ```
    
    ```
    ## [1] "numeric"
    ```
    
    ```r
    p <- x >= 4
    p
    ```
    
    ```
    ## [1] FALSE FALSE  TRUE  TRUE  TRUE
    ```
    
    ```r
    mode(p)
    ```
    
    ```
    ## [1] "logical"
    ```
    
    ```r
    q <- c("a", "b", "c")
    mode(q)
    ```
    
    ```
    ## [1] "character"
    ```
]

---

### Sequences and patterned vectors

.pull-left[

- Generate a sequence:
    
    ```r
    5:1
    ```
    
    ```
    ## [1] 5 4 3 2 1
    ```

- Using the seq function:
    
    ```r
    seq(1,5,1)
    ```
    
    ```
    ## [1] 1 2 3 4 5
    ```

- Include the sequence in vector arithmetic:
    
    ```r
    5:1*x+1:5
    ```
    
    ```
    ## [1]  10.000  14.560  15.000 180.338  18.000
    ```

]

.pull-right[

- Various functions can be used to create patterned vectors.
    
    ```r
    ones <- rep(1,10)
    ones
    ```
    
    ```
    ##  [1] 1 1 1 1 1 1 1 1 1 1
    ```
    
    ```r
    even <- seq(from=2, to=20, by=2)
    even
    ```
    
    ```
    ##  [1]  2  4  6  8 10 12 14 16 18 20
    ```
    
    ```r
    trend <- 1995:2017
    trend
    ```
    
    ```
    ##  [1] 1995 1996 1997 1998 1999 2000 2001 2002
    ##  [9] 2003 2004 2005 2006 2007 2008 2009 2010
    ## [17] 2011 2012 2013 2014 2015 2016 2017
    ```
]

***

- Combine two vectors:
    
    ```r
    c(ones,even)
    ```
    
    ```
    ##  [1]  1  1  1  1  1  1  1  1  1  1  2  4  6  8 10 12 14 16 18 20
    ```

---

### Matrices

.pull-left[

- A matrix is a two dimensional array of elements.

<BR/>

`$$\underset{2\times 3}A = \begin{bmatrix}
1 & 3 & 5\\
2 & 4 & 6
\end{bmatrix}$$`

<BR/>

`$$\underset{2\times 2}B = \begin{bmatrix}
2 & 6\\
4 & 8
\end{bmatrix}$$`

<BR/>

- The dimension of a matrix are found by counting the number of rows and columns.

- A is a 2 by 3 matrix.
    
    - B is a square 2 by 2 matrix.
    
]

.pull-right[

- Creating a matrix in R:
    
    ```r
    A <- matrix(1:6, nrow=2, byrow = FALSE)
    A
    ```
    
    ```
    ##      [,1] [,2] [,3]
    ## [1,]    1    3    5
    ## [2,]    2    4    6
    ```
    
    ```r
    B <- matrix(c(2,4,6,8), ncol=2, byrow = FALSE)
    B
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    2    6
    ## [2,]    4    8
    ```
    
- Dimensions:
    
    ```r
    dim(A)
    ```
    
    ```
    ## [1] 2 3
    ```
    
    ```r
    dim(B)
    ```
    
    ```
    ## [1] 2 2
    ```
    
]

---

### Transpose of a matrix

.pull-left[

-  The transpose of a matrix takes each row and makes it a column. That is, it switches the row and column indices.

`$$\underset{2\times 3}A = \begin{bmatrix}
1 & 3 & 5\\
2 & 4 & 6
\end{bmatrix}$$`

- The transpose of `$A$` is `$A^{\prime}$`:

`$$\underset{3\times 2}A^{\prime} = \begin{bmatrix}
1 & 2\\
3 & 4\\
5 & 6
\end{bmatrix}$$`

]

.pull-right[

- Recall matrix A:
    
    ```r
    A
    ```
    
    ```
    ##      [,1] [,2] [,3]
    ## [1,]    1    3    5
    ## [2,]    2    4    6
    ```

- Transpose of matrix A:
    
    ```r
    t(A)
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    1    2
    ## [2,]    3    4
    ## [3,]    5    6
    ```

]

---

### Subsetting and adding to a matrix

.pull-left[
- Subset a matrix the same way that we subset a vector.
    
    ```r
    A
    ```
    
    ```
    ##      [,1] [,2] [,3]
    ## [1,]    1    3    5
    ## [2,]    2    4    6
    ```
    
    ```r
    A[2,3]
    ```
    
    ```
    ## [1] 6
    ```

- To subset rows 1 and 2, and columns 1 and 3:
    
    ```r
    A1 <- A[1:2,c(1,3)]
    A1
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    1    5
    ## [2,]    2    6
    ```
    
    ```r
    A[,-2]
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    1    5
    ## [2,]    2    6
    ```

]
 
.pull-right[
- Add a row to a matrix:
    
    ```r
    rbind(A, c(10,20, 30))
    ```
    
    ```
    ##      [,1] [,2] [,3]
    ## [1,]    1    3    5
    ## [2,]    2    4    6
    ## [3,]   10   20   30
    ```
    
- Add a column to a matrix:
    
    ```r
    cbind(A, c(10,20))
    ```
    
    ```
    ##      [,1] [,2] [,3] [,4]
    ## [1,]    1    3    5   10
    ## [2,]    2    4    6   20
    ```

- Or join two matrices:
    
    ```r
    cbind(A, A1)
    ```
    
    ```
    ##      [,1] [,2] [,3] [,4] [,5]
    ## [1,]    1    3    5    1    5
    ## [2,]    2    4    6    2    6
    ```
]

---

### Element-wise arithmetic

- Addition and subtraction of matrices is always done element by element.

- Multiplication and division may be done element wise. Element by element multiplication of two matrices of equal dimensions is known as the Hadamard Product.

- A conformable matrix is a matrix with the correct dimensions for a certain matrix operation.

- Conformable matrices for element-wise operations have the same dimensions.
    
- For example, addition of the two matrices `$A$` and `$B$` is performed element-wise:

<BR/>

`$$\underset{2\times 3} A = \begin{bmatrix}
a_{11} & a_{12} & a_{13}\\
a_{21} & a_{22} & a_{23}
\end{bmatrix} \quad
\underset{2\times 3} B = \begin{bmatrix}
b_{11} & b_{12} & b_{13}\\
b_{21} & b_{22} & b_{23}
\end{bmatrix}\\[24pt]
A + B = \begin{bmatrix}
a_{11}+b_{11} & a_{12}+b_{12} & a_{13}+b_{13}\\
a_{21}+b_{21} & a_{22}+b_{22} & a_{23}+b_{23}
\end{bmatrix}\\$$`

- Subtraction is performed in the same way.

---

### Element-wise arithmetic

.pull-left[
- Consider matrices A and B:
    
    ```r
    A1
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    1    5
    ## [2,]    2    6
    ```
    
    ```r
    B
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    2    6
    ## [2,]    4    8
    ```
    
- Element by element addition:
    
    ```r
    A1 + B
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    3   11
    ## [2,]    6   14
    ```

- Element by element subtraction:
    
    ```r
    A1 - B
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]   -1   -1
    ## [2,]   -2   -2
    ```
]

.pull-right[
- Element by element multiplication:
    
    ```r
    A1 * B
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    2   30
    ## [2,]    8   48
    ```

- Element by element division:
    
    ```r
    A1 / B
    ```
    
    ```
    ##      [,1]      [,2]
    ## [1,]  0.5 0.8333333
    ## [2,]  0.5 0.7500000
    ```

- We can also multiply a matrix by a scalar:
    
    ```r
    3 * A1
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    3   15
    ## [2,]    6   18
    ```
]

---

### Vector dot product

- The dot product of a `$n \times 1$` vector `$a$` with another `$n \times 1$` vector `$b$` is:

`$$a^{\prime}b = \begin{bmatrix} a_{1} & a_{2} & \dots & a_{n} \end{bmatrix}\begin{bmatrix}
b_{1}\\
b_{2}\\
\vdots\\
b_{n}
\end{bmatrix} =
a_{1}\times b_{1} + a_{2}\times b_{2} + \dots + a_{n}\times b_{n}$$`

- The dot product results in a scalar.

- Note that `$a^{\prime}b$` equals `$b^{\prime}a$`.

- Two vectors are said to be orthogonal if their dot product equals zero.

- Example:

`$$\begin{bmatrix} 1 & 2 & 3 \end{bmatrix}\begin{bmatrix}
4\\
5\\
6
\end{bmatrix} =
1\times 4 + 2\times 5 + 3\times 6 = 4 + 10 + 18 = 32$$`

---

### Matrix multiplication

- Two matrices `$A$` and `$B$` can be multiplied if they are conformable for multiplication.

- To be conformable, the number of columns of A must equal the number of rows of B.
    
    - If `$A$` is `$r\times n$` and `$B$` is `$n\times c$` then `$AB$` will be `$r\times c$`.
    
    - Element `$jk$` of the product AB is equal to the dot product between row `$j$` of `$A$` and column `$k$` of `$B$`. For example:
    
`$$\underset{2\times 2} A = \begin{bmatrix}
a_{11} & a_{12}\\
a_{21} & a_{22}
\end{bmatrix} \quad
\underset{2\times 3} B = \begin{bmatrix}
b_{11} & b_{12} & b_{13}\\
b_{21} & b_{22} & b_{23}
\end{bmatrix}\\[24pt]
\underset{2\times 3}{AB} = \begin{bmatrix}
a_{11}\times b_{11}+a_{12}\times b_{21} & a_{11}\times b_{12}+a_{12}\times b_{22} & a_{11}\times b_{13}+a_{12}\times b_{23}\\
a_{21}\times b_{11}+a_{22}\times b_{21} & a_{21}\times b_{12}+a_{22}\times b_{22} & a_{21}\times b_{13}+a_{22}\times b_{23}
\end{bmatrix}\\$$`
    
- Note that in general, the order of matrix multiplication matters, `$AB\neq BA$`.

- In the example above, we say `$A$` is post-multiplied by `$B$` or `$B$` is pre-multiplied by `$A$`.
    
- The transpose of a matrix product is equal to the product of the transposed matrices in reverse order, `$(AB)^{\prime} = B^{\prime}A^{\prime}$`.

---

### Matrix multiplication

.pull-left[
- Recall:
    
    ```r
    A
    ```
    
    ```
    ##      [,1] [,2] [,3]
    ## [1,]    1    3    5
    ## [2,]    2    4    6
    ```
    
    ```r
    A1
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    1    5
    ## [2,]    2    6
    ```
    
    ```r
    B
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    2    6
    ## [2,]    4    8
    ```
]

.pull-right[
- `$A1\times B$`
    
    ```r
    A1 %*% B
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]   22   46
    ## [2,]   28   60
    ```

- `$A\times B$`
    
    ```r
    A %*% B
    ```
    
    ```
    ## Error in A %*% B: non-conformable arguments
    ```

- `$A^{\prime}\times B$`
    
    ```r
    t(A) %*% B
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]   10   22
    ## [2,]   22   50
    ## [3,]   34   78
    ```

]

---

### Trace and determinant

.pull-left[

- The trace of a square matrix is the sum of the diagonal elements.

`$$\underset{2\times 2} A = \begin{bmatrix}
a_{11} & a_{12}\\
a_{21} & a_{22}
\end{bmatrix}$$`
`$$tr(A) = a_{11} + a_{22}$$`

- For example:
    
    ```r
    A1
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    1    5
    ## [2,]    2    6
    ```
    
    ```r
    fBasics::tr(A1)
    ```
    
    ```
    ## [1] 7
    ```
    
    ```r
    sum(diag(A1))
    ```
    
    ```
    ## [1] 7
    ```

]

.pull-right[
- The determinant of a square matrix characterises some of the properties of the matrix.

`$$det(A) = \begin{vmatrix} A \end{vmatrix} = a_{11}\times a_{22} - a_{12}\times a_{21}$$`

- For a `$3\times 3$` matrix:

`$$E = \begin{bmatrix} 
a & b & c\\
d & e & f\\
g & h & i
\end{bmatrix}\\[12pt]
\begin{vmatrix} E \end{vmatrix} = a \begin{vmatrix} 
e & f\\
h & i\end{vmatrix} \boldsymbol{\color{red} -} b \begin{vmatrix} 
d & f\\
g & i\end{vmatrix} + c \begin{vmatrix} 
d & e\\
g & h\end{vmatrix}$$`

```r
E<-matrix(c(7,9,4,2,5,6,1,3,8),nrow = 3)
E
```

```
##      [,1] [,2] [,3]
## [1,]    7    2    1
## [2,]    9    5    3
## [3,]    4    6    8
```

```r
det(E)
```

```
## [1] 68
```

]

---

### Inverse of a Matrix

.pull-left[
- The inverse of a `$2\times 2$` matrix is:
`$$A^{-1} = 
\frac{1}{\begin{vmatrix}A\end{vmatrix}}
\begin{bmatrix}
a_{22} & -a_{12}\\
-a_{21} & a_{11}
\end{bmatrix}$$`

- In general, only square matrices with a non-zero determinant are invertible.
    
    ```r
    A1
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    1    5
    ## [2,]    2    6
    ```
    
    ```r
    solve(A1)
    ```
    
    ```
    ##      [,1]  [,2]
    ## [1,] -1.5  1.25
    ## [2,]  0.5 -0.25
    ```

]

.pull-right[
- A matrix multiplied by its inverse gives the identity matrix.
    
    ```r
    A1 %*% solve(A1)
    ```
    
    ```
    ##      [,1] [,2]
    ## [1,]    1    0
    ## [2,]    0    1
    ```
    
- `$3\times 3$` example:
    
    ```r
    E
    ```
    
    ```
    ##      [,1] [,2] [,3]
    ## [1,]    7    2    1
    ## [2,]    9    5    3
    ## [3,]    4    6    8
    ```
    
    ```r
    solve(E) %>% round(3)
    ```
    
    ```
    ##        [,1]   [,2]   [,3]
    ## [1,]  0.324 -0.147  0.015
    ## [2,] -0.882  0.765 -0.176
    ## [3,]  0.500 -0.500  0.250
    ```

]
---

### Patterned Matrices

.pull-left[
- Create a diagonal matrix:
    
    ```r
    diag(1,4,4)
    ```
    
    ```
    ##      [,1] [,2] [,3] [,4]
    ## [1,]    1    0    0    0
    ## [2,]    0    1    0    0
    ## [3,]    0    0    1    0
    ## [4,]    0    0    0    1
    ```
]

.pull-right[
- Extract the diagonal elements of a matrix:
    
    ```r
    diag(A1)
    ```
    
    ```
    ## [1] 1 6
    ```
]

***

- More complex patterned matrices can be created using the methods we covered, and these are often used within financial econometric analyses. For example:
    
    ```r
    diag(rep(c(1,2),c(6,6)))
    ```
    
    ```
    ##       [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12]
    ##  [1,]    1    0    0    0    0    0    0    0    0     0     0     0
    ##  [2,]    0    1    0    0    0    0    0    0    0     0     0     0
    ##  [3,]    0    0    1    0    0    0    0    0    0     0     0     0
    ##  [4,]    0    0    0    1    0    0    0    0    0     0     0     0
    ##  [5,]    0    0    0    0    1    0    0    0    0     0     0     0
    ##  [6,]    0    0    0    0    0    1    0    0    0     0     0     0
    ##  [7,]    0    0    0    0    0    0    2    0    0     0     0     0
    ##  [8,]    0    0    0    0    0    0    0    2    0     0     0     0
    ##  [9,]    0    0    0    0    0    0    0    0    2     0     0     0
    ## [10,]    0    0    0    0    0    0    0    0    0     2     0     0
    ## [11,]    0    0    0    0    0    0    0    0    0     0     2     0
    ## [12,]    0    0    0    0    0    0    0    0    0     0     0     2
    ```

---

## Linear regression model

- The linear regression model in matrix form. Consider as the unknown "true" model that we want to know about.

$$
    y = X\beta + \epsilon
    $$

- Where `$y$` is the dependent variable, `$X$` contains explanatory variables, `$\beta$` contains the parameters to be estimated and `$\epsilon$` is a stochastic disturbance or random error term.

<BR/>

`$$\underset{n\times 1}y = \begin{bmatrix}
Y_1\\
Y_2\\
\vdots\\
Y_n
\end{bmatrix}
,\;
\underset{n\times (k+1)}X = \begin{bmatrix}
1 & X_{11} & X_{12} & \dots & X_{k1}\\
1 & X_{21} & X_{22} & \dots & X_{k2}\\
\vdots\\
1 & X_{n1} & X_{n2} & \dots & X_{kn}
\end{bmatrix}
,\;
\underset{(k+1)\times 1}\beta = \begin{bmatrix}
\beta_1\\
\beta_2\\
\vdots\\
\beta_k
\end{bmatrix}
,\;
\underset{n\times 1}\epsilon = \begin{bmatrix}
\epsilon_1\\
\epsilon_2\\
\vdots\\
\epsilon_n
\end{bmatrix}$$`
<BR/>

- We have `$n$` observations of the dependent variable, `$n$` observations of `$k$` explanatory variables, an intercept term (the column of ones in `$X$`), and `$k+1$` parameters (the intercept plus the coefficients of our `$k$` explanatory variables).

---

## Linear regression model - example

- Assume a model with an intercept, two explanatory variables and we have 30 observations.

`$$y_i = \beta_1 + \beta_2 X_{i1} + \beta_3 X_{i2} + \epsilon_i \quad i=1,\dots,30$$`
    
- Written in matrix form:

$$
    y = X\beta + \epsilon
    $$

<BR/>

`$$\underset{30\times 1}y = \begin{bmatrix}
Y_1\\
Y_2\\
\vdots\\
Y_{30}
\end{bmatrix}
,\quad
\underset{30\times 3}X = \begin{bmatrix}
1 & X_{11} & X_{12}\\
1 & X_{21} & X_{22}\\
\vdots\\
1 & X_{301} & X_{302}
\end{bmatrix}
,\quad
\underset{3\times 1}\beta = \begin{bmatrix}
\beta_1\\
\beta_2\\
\beta_3
\end{bmatrix}
,\quad
\underset{30\times 1}\epsilon = \begin{bmatrix}
\epsilon_1\\
\epsilon_2\\
\vdots\\
\epsilon_{30}
\end{bmatrix}$$`

---

## Ordinary Least Squares Estimation

- Our aim is to generate the best possible estimates of the parameters, `$\beta$`.

- Our estimate of `$\beta$` is denoted `$\hat\beta$` and the estimate of the error term is the residual `$e$`. The residual contains what our model does not explain about `$y$`.

`$$e=y-X\hat\beta$$`

- Ordinary Least Squares (OLS) estimation simply says select the `$\hat\beta$` that minimises what our model (in particular, `$X\hat\beta$`) does not explain about our dependent variable `$y$`.

- We do this by finding the `$\hat\beta$` that minimises the sum of the squared residuals, `$e^\prime e$`.
    
    - Note that the *fitted values* of our regression are `$\hat y = X\hat\beta$`.
    
    - Next we derive the Ordinary Least Squares Estimator (OLSE). 
    
---

## Ordinary Least Squares Estimation

- Sum of the squared residuals, `$e^{\prime}e$`:

`$$\begin{align}\underset{1\times n}{e^{\prime}} \times \underset{n\times 1}{e} &=
\begin{bmatrix}
e_1\: e_2\: \dots\: e_n
\end{bmatrix}
\begin{bmatrix}
e_1\\
e_2\\
\dots\\
e_n
\end{bmatrix}\\[9pt]
&=\underset{1\times 1}{\begin{bmatrix}
e_1\times e_1\, +\, e_2\times e_2\, +\, \dots,\, +\, e_n\times e_n
\end{bmatrix}}\end{align}$$`

- Write the sum of the squared residuals in terms of our data and the parameter estimates.

- Note that the transpose of a scalar is a scalar, so `$\hat{\beta^{\prime}}X^{\prime}y=y^{\prime}X\hat\beta$`.

`$$\begin{align}e^{\prime}e &= {(y-X\hat\beta)}^{\prime}(y-X\hat\beta)\\
&= y^{\prime}y - \hat{\beta^{\prime}}X^{\prime}y - y^{\prime}X\hat\beta +\hat{\beta^{\prime}}X^{\prime}X\hat\beta\\
&= y^{\prime}y - 2\hat{\beta^{\prime}}X^{\prime}y + \hat{\beta^{\prime}}X^{\prime}X\hat\beta\end{align}$$`

---

## Ordinary Least Squares Estimation

- To find the `$\hat\beta$` that minimises the sum of the squared residuals, differentiate `$e'e$` with respect to `$\hat\beta$` and set equal to zero.

`$$\begin{align}\frac{\partial e^{\prime}e}{\partial \hat\beta} &= -2X^{\prime}y + 2X^{\prime}X\hat\beta = 0
\end{align}$$`

`$$X^{\prime}X\hat\beta = X^{\prime}y$$`

- Pre-multiply each side by `${(X^{\prime}X)}^{-1}$`:

`$${(X^{\prime}X)}^{-1}X^{\prime}X\hat\beta={(X^{\prime}X)}^{-1}X^{\prime}y$$`

- The OLSE of `$\beta$` is `$\hat\beta$`:

`$$\hat\beta={(X^{\prime}X)}^{-1}X^{\prime}y$$`

---

### Finance terms and concepts

- We will analyse the Japanese stock market in this course.

- For now we will briefly cover (or revise for anyone who has taken finance courses) some basic terms and concepts related to the stock market that will serve as background for the topics we cover next.

- Definition and characteristics of stocks.
    
    - Basic stock valuation approaches.
    
    - The Efficient Markets Hypothesis.
    
- The topics we cover after this session relate to the "cross-section of stock returns" for Japanese companies. Basic questions include:

- What explains the different returns on different companies?
    
    - Are stock returns predictable?
    
    - What variables predict stock returns?
    
- Finance contains a lot of difficult to understand jargon. Please ask me if you do not understand a term.

- [Investopedia](https://www.investopedia.com/) is a good place to look up finance term definitions.

---

### Stocks (physical or cash equity securities)

- An equity security (or stock) represents an ownership interest in a corporation.

- Investors who hold equity in a firm:

- receive income in the form of dividends, which are a portion of the company's profits. Future dividends are uncertain.

- take on the business risk of the firm.

- have the right to vote on some important company decisions.

- A stock does not have a 'maturity', but exists as long as the corporation exists unless it is 'repurchased' by the issuing corporation.

- Equity may be privately or publicly traded.

- We will focus on Japanese publicly traded stocks listed on the Tokyo Stock Exchange.
    
- Publicly traded firms are required under both regulation and stock exchange listing rules to disclose information to the public about their business and financial statements.

- We will use some of this information in our analyses.

---

### Some terms

- **Stock price:** When we are looking at data, price refers to the market price for a company's stock. This is usually the closing price that is determined in the market close auction at the end of the exchange day session (15:00 for TSX). Sometimes the stock price is referred to as "market" or "market price" in the literature, to differentiate it from investors' valuations, that is, the price an investor believes a stock should trade for.

- **Revenue:** Revenue reflects the company's income from sales.

- **Earnings:** Reflects the company's profits.

- **Dividends:** The distribution of a portion of a company's earnings to its shareholders, in the form of cash or additional stock. Dividends are usually announced on a quarterly basis.

- **Repurchases:** Companies may also distribute funds to shareholders by repurchasing stock from from investors. Repurchases have become much more common over the last 15 years.

- **Fiscal year:** The year used by a firm to report its financial information. Most Japanese firms use April to March, but some firms use different fiscal year definitions. Company's usually announce interim financial information on a quarterly basis.

---

### Some terms

- **Net Tangible assets:** The physical assets owned by a company minus intangible assets and liabilities.

- **Free cash flow:** The cash flow generated by a firm minus outflows required for its normal operations (operating expenses) and for maintaining its capital assets (capital expenses). Free cash flow is useful for valuation as it is relatively difficult for companies to manipulate.

- **Risk free rate:** Many financial theories used to model equities require a risk-free rate of return, that is, the interest rate on a "risk free" security. The yield on a government T-bill or government bond is usually used to proxy for the risk-free rate.

- **Equity risk premium:** The return that stocks provide over the return on a low-risk security, usually a government bond, to compensate the investor for taking equity risk.

- **Required rate of return:** The return an investor requires to hold a particular company's stock. This return compensates the investor for the risks involved in holding that company's stock.

---

### Equity valuation

- The price of a financial security should equal the net present value of discounted future cash flows the investor expects to receive from the security.

- What price does the investor believe the security should trade at?

- If the market price is below (above) the investor's estimate, the stock is believed to be undervalued (overvalued).

- Valuation by dividend discounting:

`$$P_i = \frac{D_{i,1}}{(1+k_{i,1})} + \frac{D_{i,2}}{(1+k_{i,2})^2} + \dots + \frac{D_{i,\infty}}{(1+k_{i,\infty})^\infty}$$`

- where `$P_i$` is the price of company `$i$`'s stock, `$D_{i,t}$` is the forecast divident for year `$t$` and `$k_{i,t}$` is the return the investor requires to hold the stock, taking into account its expected risk.
    
- This can be simplified to the Gordon Growth model witht he assumption of a constant dividend growth rate, `$g_i$`, and required rate of return:

`$$P_i = \frac{D_{i,1}}{(k_i-g_i)}$$`
    
---

### Equity valuation

- Valuation using financial ratios compare financial characteristics of the firm with the current market price. "Value investors" consider firms with relatively low ratios as attractive investments. 
   
- Price to forward (or forecast) earnings per share:

`$$PE_i = \frac{P_i}{FEPS_i}$$`
- Price to book value of net tangible assets per share:

`$$PB_i = \frac{P_i}{NAPS_i}$$`

- Price to sale revenue per share:

`$$PS_i = \frac{P_i}{S_i}$$`

- Price to free cash flow per share:

`$$PCF_i = \frac{P_i}{FCF_i}$$`

---

### The Efficient Markets Hypothesis (EMH)

- The prices of securities in an informationally efficient market fully reflect all available relevant information at any point time.

- As investors attempt to capitalise on new information, security prices adjust immediately to the correct level in an unbiased manner.

- Implies it is impossible to make an abnormally high risk-adjusted return by using active investment strategies that attempt to forecast future securities returns.

- Thus investors cannot make a higher risk-adjusted return than the market portfolio.

- A higher return is possible if the investor takes higher risk, but the risk adjusted return of the market cannot be beaten.
    
- [Investopedia explanation and video](https://www.investopedia.com/insights/what-is-market-efficiency/).

---

### The Efficient Markets Hypothesis

- Efficient with respect to what *information set*?

<BR/>

---

### Price over- and under-reactions

- In practice we often observe price over- and under-reactions in financial markets.

- Does this invalidate the EMH?
    
- Not if the price reaction is unbiased (unpredictable).

- A bias toward price over- or under-reaction would imply a predictable pattern that investors could take advantage of to beat the market return.
    
    - If the price reaction is unbiased, investors are unable to take advantage of a predictable pattern to beat the market.

---

### Price over- and under-reactions

<BR/>

---

### Are stock markets efficient?

- Complex question.

- Anomalies.

- View pre-1985.

- More modern research shows more evidence for long-term return predictability from some perspectives.

---

### Nikkei NEEDS and Financial Quest

- Nikkei Economic Electronic Databank System (NEEDS) is the most comprehensive professional database for for information on Japanese listed companies.

- Corporate financial (TANSHIN report to Tokyo Stock Exchange, Yuho report to the Financial Services Agency, NIKKEI surveys and forecasts), equity market, fixed income market, commodity market, industry, regional, and macroeconomic data.

- FinancialQUEST is used to download the data. Available in the Nakajima Library and it is possible to log in remotely via user ID and password. Japanese and English versions.

- Requires Windows 10+ and Microsoft Edge to access remotely.
    
- This is a very good resource for your Capstone Seminar paper research.
    
- Brief look and overview.

- A useful reference is the [Japan Exchange Group website](https://www.jpx.co.jp/english/).

---

## References

Ang, C. S. (2021). _Analyzing Financial Data and Implementing Financial
Models Using R_. Second. Springer Texts in Business and Economics.
Cham: Springer International Publishing. ISBN: 978-3-030-64154-2. DOI:
[10.1007/978-3-030-64155-9](https://doi.org/10.1007%2F978-3-030-64155-9).
URL:
[https://link.springer.com/10.1007/978-3-030-64155-9](https://link.springer.com/10.1007/978-3-030-64155-9).

Bennett, M. J. and D. L. Hugen (2016). _Financial Analytics with R:
Building a Laptop Laboratory for Data Science_. Cambridge University
Press. ISBN: 9781107150751. DOI:
[https://doi.org/10.1017/CBO9781316584460](https://doi.org/https%3A%2F%2Fdoi.org%2F10.1017%2FCBO9781316584460).

Kleiber, C. and A. Zeileis (2008). _Applied Econometrics with R_. 1st.
New York: Springer US. ISBN: 978-0-387-77316-2. DOI:
[10.1007/978-0-387-77318-6](https://doi.org/10.1007%2F978-0-387-77318-6).
eprint: arXiv:1011.1669v3. URL:
[http://link.springer.com/10.1007/978-0-387-77318-6
http://www.springer.com/gp/book/9780387773162](http://link.springer.com/10.1007/978-0-387-77318-6
http://www.springer.com/gp/book/9780387773162).

Matloff, N. (2011). _The Art of R Programming: A Tour of Statistical
Software Design_. No Starch Press. ISBN: 9781593273842. URL:
[https://www.nostarch.com/artofr.htm](https://www.nostarch.com/artofr.htm).

Pfaff, B. (2008). _Analysis of Integrated and Cointegrated Time Series
with R_. Second. New York: Springer-Verlag. ISBN: 978-0-387-75966-1.
URL: [papers2://publication/uuid/2D6BDE05-D712-4D3C-99CD-72EADA215C83
http://www.springer.com/gp/book/9780387759661](papers2://publication/uuid/2D6BDE05-D712-4D3C-99CD-72EADA215C83
http://www.springer.com/gp/book/9780387759661).

---

Tsay, R. S. (2014). _Multivariate time series analysis : with R and
financial applications_. Wiley. ISBN: 9781118617908.

---

## Appendix

- Locale issues:

```r
Sys.setlocale("LC_ALL", "en_US.UTF-8")
```

---

### Extra screen rec.

- [Amazon](https://www.amazon.co.jp/dp/B07ZNTHZHZ)